Capacity of Markov Channels: a Stochastic Control Approach

The problem of maximizing the input-output mutual information rate for a Markov channel is cast as a problem of controlling a partially observed controlled Markov chain on a suitable state space with time-averaged reward. This establishes the equivalence of its channel capacity with the value of a certain dynamic program. 1. Introduction. The problem of characterizing and computing the capacity of a Markov channel, i.e., a channel whose transmission characteristics vary according to a finite state Markov chain, has attracted considerable attention in recent years [8], [10], [13]. In this article, we propose a radically different approach to the problem. This is based on converting it to an equivalent stochastic control problem which allows us to characterize the channel capacity as the value of an associated dynamic program. This is achieved by adapting the techniques of [3] for the present case. In addition to the theoretical interest, this also opens up the possibility of employing numerical techniques of stochastic control for approximate capacity calculations. The paper is organized as follows: The rest of this section sets up the notation and introduces the problem. The next section derives the equivalent stochastic control problem. Section 3 derives the associated dynamic programming equations. Section 4 outlines some extensions and directions for future research. Two technical appendices deal with resp. the lengthy proof of a key lemma (Lemma 3.2) and a scheme for relaxing one of the main modelling restrictions (aperiodicity of the underlying chain). We shall be dealing with the following objects: • An irreducible aperiodic Markov chain {S n } taking values in a finite state space S, indicating the randomly varying channel characteristics. • An input process {X n } taking values in a finite input alphabet X. • An output process {Y n } taking values in a finite output alphabet Y.